Evaluating Inter-Organizational Information Systems

This chapter describes different means of evaluating the usability and suitability of computer-based inter-organizational information systems (IOISs). It begins with describing why doing so is important yet difficult, and provides an assessment of the advantages and disadvantages of the major types of evaluation. It continues with a case study focusing on determining whether an application provides the necessary insight into other collaborators’ identities, presence, and activities while keeping sensitive information private from a subset of the collaborators. The goal of this chapter is to provide practical guidance to organizations seeking IOISs to help them choose (or develop) an IOIS that best meets their needs. Evaluating Inter-Organizational Information Systems 267 Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited. Introduction Information technology can be used to support collaborations and partnerships among organizations for competitive purposes. Organizations have developed the notion of inter-organizational systems (IOSs), also known as inter-organizational information systems (IOISs), to support these collaborations and partnerships. An IOIS is defined as an automated information system shared by two or more organizations (Cash & Konsynski, 1985) in a collaborative fashion. Compare the definition of an IOIS with the definition of computer-supported collaborative work (CSCW) applications: applications that support coordinated activity carried out by groups of collaborating individuals (Greif, 1988). CSCW applications are also known as multiuser, groupware, or collaborative applications. Collaborative applications normally provide capabilities beyond simple information access to facilitate communication and collaboration among the partners. Depending upon the collaborative application, both synchronous and asynchronous communications may be supported, and documents can be shared. Some collaborative applications incorporate video to support communications and negotiations. These coordination mechanisms are essential to efficient collaboration among cooperating organizations. In fact, because IOISs are computerbased systems used to collaborate across organizations, they are a subset of collaborative applications. Hong and Kim (1998) built on Cash and Konsynski’s (1985) work by developing a framework for classifying the various types of IOISs. Their classification scheme is based on three categories: vertical linkage, horizontal linkage, and cross linkage. Vertical systems connect suppliers with sellers with the goal of more efficient marketing. This type of system gives sellers, for example, the capability to place orders quickly and gives suppliers sales data to help them plan production. Horizontal systems link homogeneous groups of businesses. Partnerships within an industry, often consisting of smaller businesses, benefit from improved access to information. Cross systems are an attempt to integrate horizontal and vertical links into one complete system. It is necessary to understand the roles of the participants or collaborators in IOISs in order to provide the necessary system capabilities to support a variety of tasks. For example, consider a vertical IOIS that links a manufacturer with a number of suppliers. A subset of those suppliers may be competitors who are negotiating terms with the manufacturer. Suppliers may want to use this system to share contractual information with the manufacturer but not with each other. Vertical and cross IOISs will need to support the most diverse set of users (e.g., suppliers, manufacturers, and retailers), though horizontal IOISs might also need to support differing groups of collaborators (e.g., manufacturers from the 268 Drury and Scholtz Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited. Eastern United States versus manufacturers from the Western United States). The roles of participants and their different information sharing needs should be taken into account when evaluating which IOIS is appropriate for a set of cooperating organizations. To help organizations evaluate which IOIS they should adopt, and provide guidance for developing an IOIS, this chapter includes an assessment of the advantages and disadvantages of using different types of evaluation methods for determining the suitability of IOIS applications. The other contributions of this chapter are as follows: • An explanation of why IOISs are difficult, yet important, to evaluate • A description of how the Synchronous Collaborative Awareness and Privacy (SCAPE) awareness framework could be used to evaluate an IOIS application • A case study of evaluating the Groove1 application’s suitability for use by a collaborating team that includes members from organizations with different goals